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(57) Abstract 

The current frame of the picture is com- 
pared (2) block-by-block with the previous frame 
to identify changed picture elements (pixels). The 
resulting matrix is matched ("vector quantised") 
(4) to one of a set of standard matrices ("VQ 
shapes"). Only those pixels flagged by the select- 
ed VQ shape are transmitted plus a "VQ shape 
code". This technique is modified in that the cur- 
rent frame is also compared (30) with a "back- 
ground" frame stored (31) at the transmitter and 
at the receiver. Where, for any block, all the pix- 
els flagged by the VQ shape are indicated by the 
second comparison as being the same as the 
background, the pixels are not sent - instead a 
shape code plus a "copy background" instruction 
is transmitted. 
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SISNRL C0DIH6 

The present invention relates to coding of video 
signals, especially using conditional replenishment 
coding, where information is generally transmitted only m 
resoect of elements of a frame of the picture which have 
changed relative to a previous frame? the transmitted data 
being used at a receiver to update a stored version of the 
picture. 

Thus Picture elements of each block of an image to be 
coded are compared with those of the corresponding block 
of a previously coded image to determine whether the block 
has changed between the two images, if so, picture element 
data are generated for output. 

Such a system is described in international patent 
application polished under no. W086/03922, which also 
proposes that" the block be compared with the corresponding 
Llock of an earlier (reference) image. If they are deeme„ 
to be the same, no picture data are generated - instead, a 
codeword is produced to indicate that the receiver is to 
obtain its data from a locally stored replica of ue 
reference image, 

^cording to one aspect of the present inversion, 
there is provided a method of coding an image comprising, 
for each of a plurality of blocks of an image: 
(i) comparing picture elements of the block with those 
of the corresponding block of a previously coded 
imaae to produce a matrix of values each 
indicating whether the corresponding element is,, 
in accordance with a predetermined criterion, 
deemed to have changed between the two images; 
(ti) matching the matrix to one of a predetermined set 
of such matrices each of which identifies a region 
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of the block as being deemed to have changed, and 
generating a codeword identifying that one matrix 
of the set? 

(iii) comparing picture elements of the block with those 
of the corresponding block of a reference image 
composed of blocks- from at least one earlier frame 
to determine whether' the block is, in accordance 
with a predetermined criterion, deemed to have 
changed between the two images? 

(iv) coding for output those elements within the 
identified region, unless all the elements within 
the region are identified by comparison step (iii) 
as being unchanged relative to the reference 
image, whereupon a codeword indicating this is 
generated.. 

In another aspect, the invention provides an apparatus 
for coding an image comprising, 

.(i) . means for comparing, for each of a plurality of 
clocks cf an image, picture elements of the block 
with those of the corresponding block of a 
previously coded image to produce a matrix of 
values each .indicating whether the corresponding 
element is r in ' accordance with a predetermined 
criterion, deemed to have changed between the two 
images } 

(ii) means for matching the matrix to one of a 
predetermined set of such matrices each of which 

. identifies a region of the block as being deemed 
to have changed, and generate a codeword, 
identifying that one matrix of the set? 

(iii) .a store arranged to store a reference image 

composed of blocks from at least cue earlier frame 
and means for comparing picture elements cf the 
block with those cf the corresponding block cf the 
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reference image to determine whether the block is, 
in accordance with a predetermined criter.cn, 
deemed to have changed between the two images f 
U v) means for coding for output those elements within 
the identified region, unless all the elements 
within the region are identified by comparison 
step (iii) as being unchanged relative to the 
reference image, whereupon a codeword indicating 
this is generated. _ 
The matching step - often termed vector quantisation - 
of two-dimensional maps has been proposed previously for 
picture coding, for example in European patent application 
serial no. 023901a, where blocks of transform coefficient 
blocks are classified. 

The present invention, however, further exploits the 
vector quantisation, in that not only can the 
classification be used (if desired) to reduce the amoun. 
of information that has to be transmitted to indicate 
which elements have been coded for output, but also tn. 
number of occasions on which a "reference" codeword is 
instead generated are increased. 

some embodiments of the invention wii, no, be 
described, by way of example, with reference tc tae 
accompanying drawings, in which: 

- Figure 1 is a block diagram of a coder according to 
one embodiment of the invention; 

- Figure 2 shows a typical bit map produced by the 
chanae detector of the coder of Figure 1; 

1 Figure 3 illustrates a few standard tit patterns 
used by the vector quantiser of the coder of Figure 1; and 

- Figure 4 is a block diagram of a decoder according 
to another embodiment of the present invention. 

Figure X shows a conditional replenishment video 
coder, where video signals (assumed to be in signal form) 
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are supplied to an input 1. The current frame of a 
picture is compared in a change or movement aetector 2 
with the output of a local decoder 3. The local decoder 
produces a "previous frame" output which is the same as 
5 that produces by a remote decoder; the object of the 

comparison is to identify those parts of the picture which 
have changed so that only information concerning those 
parts needs- to be sent to the decoder, to update a stored 
representation of the frame. 
10 The signals are processed on a block-by~block basis - 

an 8 x . 8 block is assumed - and the change detector 2 
therefore produces an 8 x 8 bit map. A typical map is 
shown in figure 2, where the elements of the block 
corresponding to picture elements (pixels) which have 
IS changed are shown shaded. 

In practice, transmission of information concerning 
onl y the changed elements involves a significant 
addressing overhead and therefore it is preferred to match 
the bit map to one of a limited number - typically forty - 
20 of ' standard shapes (a few are illustrated in figure 3). 

Since it is preferable to transmit information for an 
unchanged pixel than to fail to transmit information for a 
changed pixel, the shape chosen is the smallest {i.e. 
with the least number of shaded elements) which has a 
25 shaded area covering the shaded elements of the bit map. 

This process is termed vector quantisation (VQ) and is 
indicated as vector quantizer 4 in figure 1. One 
transmits, for the block, a VQ number identifying the 
chosen shape, along with information concerning pixels 
30 deemed - in accordance with the chosen shape - to have 

changed. The output data are combined and buffered in a 
buffer 5 prior to passage to an output 6. is is 
conventional in such systems, the buffer is used to smooth 
variations in the rate at which data are generated (due to 
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the picture content dependent coding) and interface to a 
regular transmitted rate and the buffer fullness state 
usea to control the rate of generation (e.g. by varying 
the change detector thresholds (control line 7)). 

The pixels deemed to have changed are coded in an 
encoder 8 (for example in the manner described in our 
international patent application PCT/GB88/00709 

(publication no ) and European patent 

application no. 88307981.6 (publication no ). 

Many televised scenes - especially in a 
videoconference or videotelephone environment - contain 
moving persons or objects set against a fixed background. 
Pixels observed as changed by the change detector 2 will 
relate either to objects which have changed their position 
(or entered the scene) or to parts of the background 
uncovered by the object. In the present coder, a second 
change detector 30 is also shown, which compares the 
current frame with a reference or background frame stored 
in a frame store 31. Acquisition of the stored background 
frame will be discussed further below, but in figure 1 is 
assumed to have been acquired from the local decoder 3 in 
response to manual operation of a switch 32 at the 
commencement of a transmission; a code being transmitted 
to the receiver to initiate similar action at the remote 
decoder. 

The change detector 30 produces a bit map identifying 
those pixels of the current frame which are the same as 
the background. 

If this shows that the new image is different from the 
background for any of the pixels declared as changed by 
the VQ shape the background information cannot be used and 
information concerning those pixels is transmitted along 
with the VQ number. 
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If, however the changed area as given by the Vp_ shape 
covers only pixels which are identified by the detector 30 
■ as. being the same as the corresponding pixels of the 
stored background frame, then the VQ number is accompanied 
5 by a reserved 'codeword indicating "background" and no 

further information needs to be transmitted for the block 
in question. 

These functions are accomplished in figure 2 by a 
comparator 33 whose output 34 overrides the operation of 
lo the encoder 8. 

Figure 4 shows a decoder, An input processor 40 
receives the. coded input • signals. The background frame is 
stored in a background store 41. In normal operation, the 
processor 40 uses the pixel information received to update 
is via line 42 the contents of a frame store 43, using the 

received VQ numbers via line 44 to control the frame store 
addressing. . When, however, it receives the reserved 
"background" codeword (via line 45), it recovers the 
relevant pixels (identified by the VQ number) from the 
20 background store 41 and enters then into the frame store 

43. This is illustrated, schematically by a changeover 
switch 46, The frame store 43 is read out {by output 
control means 47). to produce the received video at an 
output 4B. As in the coder, the background store 41 is 
25 loaded from "the decoded image in store 43 when a 'load 

background 1 instruction is received (iine 49). 

Hote that the- local decoder 3 of Figure 1 can be of 
the same construction as the decoder of Figure 4, 
although,, of course, in practice the local decoder would 
30 use the background store 31 rather than contain one of its 
own. 

The change detectors 2,30 can in principle be any 
conventionally used, or may both be as described in our 
above-mentioned patent applications. The vector 
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quantiser 4 can again be a known device. One possibility 
is that described in our International patent application 
no. PCT/GB87/00B16 (publication no. W08B/04084) and 
European patent application no. 8527787 (publication no. 
0272794). 

Simpler but cruder, another option is to convert the 
8x8 matrix to a 4x4 matrix by creating each 'new' element 
as an OR function of four 'olfl' elements; this reduces the 
number of elements in the matrix to a size (15) which can 
be used to address a look-up table - in the form of a 64 
Kbyte read only memory in which the appropriate VQ numbers 
are stored. 

The background scene may be 'frozen' manually at the 
commencement of a transmission. It may, but does not have 
to, consist solely of a fixed background. For example it 
may include seated figures (thereby covering the situation 
where a person momentarily passes his hand over his face 
if the face forms part of the "background" , the face does 
not have to be retransmitted). 

It may be desirable to include provision for updating 
the background. For example, the incoming video could be 
monitored' at the coder (by a unit 35 shown dotted in 
Figure 1) and parts of the picture which differ from the 
original background but have remained unchanged for a 
predetermined period of time inserted into the frame 
store, a signal being sent (via line 36) to the decoder to 
instruct it to do likewise. 

An alternative method of updating the background store 
abandons any attempt at identifying genuine background, 
but instead forms a reference image which is a composite 
of blocks taken from preceding images over a period. Each 
frame period, data for a few (e.g. eight) selected blocks 
scattered over the image area is er.tered into the 
background stores, the block selection being such that 
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different blocks are selected from each frame until the 
whole image area has been covered, assuming 1024 blocks 
per frame at 25 frames per second, this represents a 
period of approximately 40 seconds. The blocks could be 

5 loaded from the frame store 43 of the decoder, and a 

corresponding store in the local decoder 3, the change 
monitor then being replaced by a simple address generator 
35 to select the appropriate blocks. Clearly, this is 
less effective, in that some blocks of the reference image 

10 will not represent background material, but a significant 

coding advantage is still obtained, and it has the merit 
of simplicity. 

a variation ■ of this approach provides that data for 
selected blocks, instead of being drawn from the decoded 

is image, are actually transmitted - i.e. transmission of the 

whole block is forced even if only a part, or none, of it 
. is indicated by the detector 2 and quantiser 4 as moving. 
In this case, the frame stores 31, 41 take their input 
from, the input 1 and processor 40 respectively. This also 

20 has the benefit of ensuring that transmission errors on 

blocks which rarely change do not persist in the decoded 
image. 
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1. A method of coding an image comprising, for each 

of a plurality of blocks of an image: 

(i) comparing picture elements of the block with 
those of the corresponding block of a previously 
coded image to produce a matrix of values each 
indicating whether the corresponding element is, 
in accordance with a predetermined criterion, 
deemed to have changed between the two images; 

(ii) matching the matrix to one of a predetermined set 
of such matrices each of which identifies a 
region of the block as being deemed to have 
changed, and generating a codeword identifying 
that one matrix of the set; 

(iii) comparing picture elements of the block with 
those of the corresponding block of a reference 
image composed of blocks from at least one 
earlier frame to determine whether the block is, 
in accordance with a predetermined criterion, 
deemed to have changed between the two images; 

(iv) coding for output those elements within the 
identified region, unless all the elements within 
the region are identified by comparison step 
(iii) as being unchanged relative to the 
reference image, whereupon a codeword indicating 
this is generated. 

2. h method according to claim 1 in which, in the 

coding of the elements within the identified region, at 
least some of the elements are coded as the difference 
between that element and a predicted value for that 
element derived from one or more previously ceded elements 



of the block, the sequence cf coding of the elements being 
dependent on the orientation of the identified region. 

3. An apparatus for coding an image comprising, 

(i) means for comparing, for each of a plurality of 
blocks of an image r picture elements of the block 
with those of the corresponding block of a 
previously coded image to produce a matrix of 
values each indicating whether the corresponding 
element is, in accordance with a predetermined 
criterion, deemed to have changed between the two 
images ; 

(ii) means for matching the matrix to one of a 
predetermined set of such matrices each of which 
identifies a region of the block as being deemed 
to have changed, and generate a codeword 
identifying that one matrix of the set? 

{iii) a store arranged to store a reference image 
composed of blocks from at least one earlier 
frame and means for comparing picture elements of 
the block with those of the corresponding block 
cf the reference image to . determine whether the 
block is, in accordance with a predetermined 
criterion, deemed to have changed between the two 
images? 

Civ) means for coding for output those elements within 
the identified region, unless all the elements 
within the region are identified by comparison 
step (iii) as being unchanged relative to ' the 
reference image, whereupon a codeword indicating 
this is. generated. 

4. M apparatus according to claim 3, including 

means for comparing successive images to identify parts cf 
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the image which have remained unchanged for a 
predetermined period of time, and in the event of such 
parts being identified, to: 

(a) update the stored reference image; and 

(b) code for output data indicating which parts of 
the reference image have been thus updated. 

5. An apparatus according to claim 3, including 
means arranged in operation to effect periodic replacement 
of a minority of the blocks of the reference image stored 
in the store by the corresponding blocks of a recent 
image, the blocks being differently selected for 
successive replacements such that the store always 
contains a composite image made up of blocks from a 
plurality of frames of the image being coded. 

6. h decoder for use with the coder of claim 3 or 5, 
comprising: 

(a) . a frame store for storing a received image; 

(b) a second frame store; 

(c) control means responsive to received data to 
update the contents of the second frame store 
using the received data and responsive to a 
received codeword to update the second frame 
store with information from the first frame store; 

(d) means for repetitively reading the contents of 
the second frame store to produce a video output 
signal. 
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